from alarms import *
# print(pio.renderers)
pio.renderers.default = "notebook"
path = "/home/waris/Github/tupras-analysis/new-alarm/stats-one-month.csv"
df = pd.read_csv("stats-one-month.csv", low_memory=False)
df["StartTime"] = df["StartTime"].apply(lambda d: parse(d))
df["EndTime"] = df["EndTime"].apply(lambda d: parse(d))
df["Time"] = df["Time"].apply(lambda d: datetime.strptime(d,"%H:%M").time())
# for col in df.columns:
# print(col, type(df[col][0]))
# print("===============")
# df.info()
df.head(20)
# df.describe()
The following graph shows all the alarms triggered in this dataset. The x-axis represents the activation time of an alarm, and the y-axis shows the duration (i.e., TimeDelta= StartTime - EndTime) of the corresponding activation. As we can see that most of the alarms deactivated within 20 seconds (it will be more clear in followings sections).
fig = px.scatter(df, x="StartTime", y="TimeDelta",render_mode="webgl")
fig.show()
This is one of the most important box plots in the whole analysis. It will help us to determine the threshold for TimeDelta. If the duration between activation and deactivation (i.e., TimeDelta = Deactivation - Activation) is less than the threshold then we will not transmit such alarm to the historian. From the following box plot, we can see that the first quartile (q1) is equal to 16 seconds which means if we set the threshold constant equals to 16 seconds we will directly reduce 25% of the communications between the DCS systems and Historian server. Similarly, if we set threshold constant to 29 seconds (i.e., q2 or median value) the 50 % of the storage & and communication will be reduced.
Questions:
Note: If hover the mouse over any graph it will show the values.
fig = px.box(df, y="TimeDelta")
fig.update_yaxes(range=[0, 100])
fig.show()
From the first histograms, we can see that "47TI931" triggered the most number of alarms. Additionally, all the alarms are related to “IOP” condition.
Questions
fig = px.histogram(df, x = "SourceName")
fig.update_layout(title="Number of times each SourceName triggered an alarm.")
fig.show()
fig = px.histogram(df, x = "SourceName", color="Condition", barmode='group')
fig.update_layout(title="Number of times a condition occured for a SourceName.")
fig.show()
fig = px.box(df, x= "SourceName", y="TimeDelta")
fig.update_yaxes(range=[0, 200])
fig.update_layout(title="Box Plot of TimeDelats (Deactivaion - Activation) for each SourceName")
fig.show()
As we can see from the following graphs, the most frequent alarms are related to communication problems. For instance, IOP condition occurred more 191 thousand time. Additionally, from the box plot, we can see that that durations (i.e., timedelta = deactivation time - activation time) of IOP alarms are shorter.
Conclusion: IOP communications can discard easily as it just represent only the communication issue between the field device and the DCS system. So if we discard such alarms we reduce the communication between the DCS system and historian server to a few hundreds instead of thousands.
Questions:
fig = px.histogram(df, x = "Condition")
fig.show()
fig = px.box(df, x= "Condition", y="TimeDelta")
fig.update_yaxes(range=[0, 200])
fig.show()
fig = px.histogram(df, x="WeekDay", color="Condition", barmode='group')
fig.show()
sources = {} # contains each unique sourcename as key and corresponding alarms in a list
for sname in df["SourceName"].unique():
sources[sname] = []
temp_dict = None
for i in range(df.shape[0]):
temp_dict = {}
for key in df.columns:
temp_dict[key] = df[key][i]
sources[df["SourceName"][i]].append(temp_dict)
If an alarm from the same source is triggered 3 times or less then such situation of chattering is considered normal. However, from the following the histograms, we can see that thousands of alarms are triggered more than thrice in most of the sensors. For instance, consider "47TI931A", it chatters 6 times in a minute more than 31 thousand times.
Note: x-axis represents the number of times an alarm chatter in the duration of one minute. The y-axis represents how many times such conditions (i.e., chatters) occur.
Conclusion: Such conditions (in which an alarm is triggered more than 3 times in a minute) are abnormals. So, if we detect such conditions at the edge it may help to do.... [list the usages].
Questions:
cols = 2
rows = math.ceil(len(sources.keys())/cols)
fig = make_subplots(rows=rows, cols=cols)
t = 0
snames = list(sources.keys())
for row in range(1, rows+1):
for col in range(1,cols+1):
chats = findChatterings(sources[snames[t]])
chats = [v["count"] for v in chats.values()]
trace = go.Histogram(x=chats,name=snames[t])
fig.add_trace(trace,row =row, col=col)
t += 1
fig.show()
In this section, I tried to find the time between 1 deactivation to next deactivation. In ideal scenario, there should be 1 alarm per minute. However, when I took the time difference between 1 deactivation to the next activation of any alarm the difference (delta) was mostly under 10 seconds which means that a huge number alarms are generated in 1 minute.
Note: The x-axis represents the time delta (i.e., the time difference between deactivation and activation of an alarm), while the y-axis represents the frequency (count) of corresponding deltas.
Questions:
alarms = []
# concatenating lists of alarms
for sname in sources.keys():
alarms = alarms + sources[sname]
freq = frequencyOfAlarmsActivated(alarms)
d = {}
for v in freq:
if v < 200:
d[v] = 0
for v in freq:
if v < 200:
d[v] +=1
counts = [v for v in d.values()]
deltas = [k for k in d.keys()]
fig = px.bar(x=deltas, y = counts)
fig.show()